{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "41c3f8f6",
   "metadata": {},
   "source": [
    "# Getting Started with Pipeline"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "db9d09ef",
   "metadata": {
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "source": [
    "## What is `Pipeline` in Towhee\n",
    "\n",
    "`Pipeline` is a pythonic computation framework for unstructured data in machine learning and data science. It provides a [method-chaining style API](https://towhee.readthedocs.io/en/main/index.html) that makes it easier for a data scientist or researcher to assemble a data processing pipeline.\n",
    "\n",
    "- Pythonic Method-Chaining Style API: Designed to behave as a python list or iterator, `Pipeline` is easy to understand for python users and is compatible with most popular data science toolkits. Function/Operator invocations can be chained one after another, making your code clean and fluent.\n",
    "\n",
    "- Feature-Rich Operator Repository: There are various pre-defined operators On the towhee hub, which cover the most popular deep learning models in computer vision, NLP, and voice processing. Using these operators in the data processing pipeline can significantly accelerate your work."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "f7ac0f05",
   "metadata": {},
   "source": [
    "## Preparation \n",
    "\n",
    "Before getting started, we need to install the dependencies and download the test data.\n",
    "\n",
    "### Install dependencies\n",
    "\n",
    "First we need to install dependencies such as towhee, pandas."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "97b26871",
   "metadata": {},
   "outputs": [],
   "source": [
    "! python -m pip install -q towhee"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "6fc95353",
   "metadata": {},
   "source": [
    "## Create a `Pipeline`"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "8441de76",
   "metadata": {},
   "source": [
    "This is the simplest example of `Pipeline`, it recieves the input and outputs the data immediately. We start with this `Pipeline` to illustrate the structure of a `Pipeline`, a `Pieline` should at least contain a `input` and a `output` node."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "bfa67b0c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table style=\"border-collapse: collapse;\"><tr><th style=\"text-align: center; font-size: 130%; border: none;\">num</th></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">1</td></tr></table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from towhee import pipe\n",
    "from towhee.datacollection import DataCollection\n",
    "\n",
    "\n",
    "p = (\n",
    "\tpipe.input('num')\n",
    "\t\t.output('num')\n",
    ")\n",
    "\n",
    "res = p(1)\n",
    "DataCollection(res).show()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "498c90b3",
   "metadata": {},
   "source": [
    "## Nodes"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "6652d0a0",
   "metadata": {},
   "source": [
    "### Map\n",
    "\n",
    "Usage:\n",
    "\t\n",
    "\tmap(input_schema, output_schema, fn, config=None)\n",
    "\n",
    "Map node get the specified data by input_schema from each element as input, apply it to a function, then specify the schema of output as output_schema.\n",
    "\n",
    "<img src=\"map_node.png\" width=\"600px\">"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "85ef02c5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table style=\"border-collapse: collapse;\"><tr><th style=\"text-align: center; font-size: 130%; border: none;\">num</th></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">2</td></tr></table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "p = (\n",
    "\tpipe.input('num')\n",
    "\t\t.map('num', 'num', lambda x: x * 2)\n",
    "\t\t.output('num')\n",
    ")\n",
    "\n",
    "res = p(1)\n",
    "DataCollection(res).show()\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "c7ce9e50",
   "metadata": {},
   "source": [
    "### Flat Map\n",
    "\n",
    "Usage:\n",
    "\n",
    "\tflat_map(input_schema, output_schema, fn, config=None)\n",
    "\n",
    "Flat Map get the specified data by input_schema from each element as input, apply it to a function, then flatten the results, and specify the schema of output as output_schema.\n",
    "\n",
    "<img src=\"flat_map_node.png\" width=\"600px\">"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "e2c23d89",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table style=\"border-collapse: collapse;\"><tr><th style=\"text-align: center; font-size: 130%; border: none;\">num</th></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">2</td></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">4</td></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">6</td></tr></table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "p = (\n",
    "\tpipe.input('num')\n",
    "\t\t.flat_map('num', 'num', lambda x: [i * 2 for i in x])\n",
    "\t\t.output('num')\n",
    ")\n",
    "\n",
    "res = p([1, 2, 3])\n",
    "DataCollection(res).show()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "fbba34b9",
   "metadata": {},
   "source": [
    "### Filter\n",
    "\n",
    "Usage:\n",
    "\n",
    "\tfilter(input_schema, output_schema, filter_columns, fn, config=None)\n",
    "\n",
    "Filter only select elements that satisfy the predicate, it will check the function with filter_columns as input. And the input_schema and out_schema are the data before and after the filter node, so their lengths are required to be equal.\n",
    "\n",
    "<img src=\"filter_node.png\" width=\"600px\">"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "dbc5a04b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table style=\"border-collapse: collapse;\"><tr><th style=\"text-align: center; font-size: 130%; border: none;\">num</th></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">4</td></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">6</td></tr></table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "p = (\n",
    "\tpipe.input('num')\n",
    "\t\t.flat_map('num', 'num', lambda x: [i * 2 for i in x])\n",
    "\t\t.filter('num', 'num', 'num', lambda x: x > 2)\n",
    "\t\t.output('num')\n",
    ")\n",
    "\n",
    "res = p([1, 2, 3])\n",
    "DataCollection(res).show()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "b9249e20",
   "metadata": {},
   "source": [
    "### Window\n",
    "\n",
    "Usage:\n",
    "\n",
    "\twindow(input_schema, output_schema, size, step, fn, config=None)\n",
    "\n",
    "Assign all the elements into windows of finite size, over which to apply computations(fn). The size of each window is configured by the size parameter, the step parameter controls how frequently a window is started. \n",
    "\n",
    "<img src=\"window_node.png\" width=\"600px\">"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "83d42f4b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table style=\"border-collapse: collapse;\"><tr><th style=\"text-align: center; font-size: 130%; border: none;\">num</th> <th style=\"text-align: center; font-size: 130%; border: none;\">nums</th></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">2</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[4,8] len=2</td></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">4</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[8,12] len=2</td></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">6</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[12,16] len=2</td></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">8</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[16] len=1</td></tr></table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "p = (\n",
    "\tpipe.input('num')\n",
    "\t\t.flat_map('num', 'num', lambda x: [i * 2 for i in x])\n",
    "\t\t.window('num', 'nums', 2, 1, lambda x: [i * 2 for i in x])\n",
    "\t\t.output('num', 'nums')\n",
    ")\n",
    "\n",
    "res = p([1, 2, 3, 4])\n",
    "DataCollection(res).show()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "13f0d796",
   "metadata": {},
   "source": [
    "### Time Window\n",
    "\n",
    "Usage:\n",
    "\n",
    "\ttime_window(input_schema, output_schema, timestamp_col, size, step, fn, config=None)\n",
    "\n",
    "Assign all the elements into time windows of finite size based on the timestamp_col column, over which we can apply computations(fn). The time interval of each time window is configured by the size parameter, and the step parameter controls the time frequency at which a time window starts. \n",
    "\n",
    "<img src=\"time_window_node.png\" width=\"600px\">"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table style=\"border-collapse: collapse;\"><tr><th style=\"text-align: center; font-size: 130%; border: none;\">num</th> <th style=\"text-align: center; font-size: 130%; border: none;\">timestamp</th> <th style=\"text-align: center; font-size: 130%; border: none;\">nums</th></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">1</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">0</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[1,2] len=2</td></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">2</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">1000</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[2,3] len=2</td></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">3</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">2000</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[3] len=1</td></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">4</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">4000</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[4] len=1</td></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">5</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">5000</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[4,5] len=2</td></tr></table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "p = (\n",
    "\tpipe.input('num', 'timestamp')\n",
    "\t\t.flat_map(('num', 'timestamp'), ('num', 'timestamp'), lambda x,y: [(i, j) for i, j in zip(x, y)])\n",
    "\t\t.time_window('num', 'nums', 'timestamp', 2, 1, lambda x: x)\n",
    "\t\t.output('num', 'timestamp', 'nums')\n",
    ")\n",
    "\n",
    "res = p([1, 2, 3, 4, 5], [0, 1000, 2000, 4000, 5000])\n",
    "DataCollection(res).show()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "94871afd",
   "metadata": {},
   "source": [
    "### Window All\n",
    "\n",
    "Usage:\n",
    "\n",
    "\twindow_all(input_schema, output_schema, fn, config=None)\n",
    "\n",
    "Assign all the elements into one window, over which we can apply computations(fn).\n",
    "\n",
    "<img src=\"window_all.png\" width=\"600px\">"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "220d16b2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table style=\"border-collapse: collapse;\"><tr><th style=\"text-align: center; font-size: 130%; border: none;\">num</th> <th style=\"text-align: center; font-size: 130%; border: none;\">nums</th></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">2</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[4,8,12,16] len=4</td></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">4</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[4,8,12,16] len=4</td></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">6</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[4,8,12,16] len=4</td></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">8</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[4,8,12,16] len=4</td></tr></table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "p = (\n",
    "\tpipe.input('num')\n",
    "\t\t.flat_map('num', 'num', lambda x: [i * 2 for i in x])\n",
    "\t\t.window_all('num', 'nums', lambda x: [i * 2 for i in x])\n",
    "\t\t.output('num', 'nums')\n",
    ")\n",
    "\n",
    "res = p([1, 2, 3, 4])\n",
    "DataCollection(res).show()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "b1bd3245",
   "metadata": {},
   "source": [
    "### Concat\n",
    "\n",
    "Usage:\n",
    "\n",
    "\tconcat(*pipeline)\n",
    "\n",
    "Concat one or more pipelines to the existing pipeline and update all data from each pipeline.\n",
    "\n",
    "<img src=\"concat_node.png\" width=\"600px\">"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "af875770",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table style=\"border-collapse: collapse;\"><tr><th style=\"text-align: center; font-size: 130%; border: none;\">a</th> <th style=\"text-align: center; font-size: 130%; border: none;\">b</th> <th style=\"text-align: center; font-size: 130%; border: none;\">c</th> <th style=\"text-align: center; font-size: 130%; border: none;\">d</th> <th style=\"text-align: center; font-size: 130%; border: none;\">e</th></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">1</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">2</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">3</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">2</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">-1</td></tr></table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from towhee import pipe\n",
    "pipe0 = (\n",
    "\tpipe.input('a', 'b', 'c')\n",
    ")\n",
    "pipe1 = (\n",
    "\tpipe0.map('a', 'd', lambda x: x+1)\n",
    ")\n",
    "\n",
    "pipe2 = (\n",
    "\tpipe0.map(('b', 'c'), 'e', lambda x, y: x - y)\n",
    ")\n",
    "\n",
    "pipe3 = pipe2.concat(pipe1).output('a', 'b', 'c', 'd', 'e')\n",
    "\n",
    "res = pipe3(1, 2, 3)\n",
    "DataCollection(res).show()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "c3bf17bd",
   "metadata": {},
   "source": [
    "## Towhee Operator\n",
    "\n",
    "Operators are the basic units of computation that can be applied to the elements within a DataCollection. There are many predefined Operators on the [Towhee hub](https://towhee.io/operators?limit=30&page=1), including popular deep learning models ranging from computer vision to natural language processing.\n",
    "\n",
    "We can load an Operator from the Towhee hub as following:\n",
    "\n",
    "\tops.namespace.operator_name(**kws, *args)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "8e38fcd1",
   "metadata": {},
   "outputs": [],
   "source": [
    "from towhee import ops\n",
    "op = ops.image_decode.cv2()\n",
    "img = op('1.jpg')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "121a949c",
   "metadata": {},
   "source": [
    "### Custom Operator\n",
    "\n",
    "It is also easy to define custom operators with standard Python functions with `towhee.register`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "fab1f328",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from towhee import register\n",
    "\n",
    "@register\n",
    "def add_1(x):\n",
    "    return x+1\n",
    "\n",
    "ops.add_1()(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8e699c40",
   "metadata": {},
   "source": [
    "If the operator needs additional initializations arguments, it can be defined as a class:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "4e9c4179",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "@register\n",
    "class add_x:\n",
    "    def __init__(self, x):\n",
    "        self._x = x\n",
    "    def __call__(self, y):\n",
    "        return self._x + y\n",
    "\n",
    "ops.add_x(x=1)(2)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "2ffc3114",
   "metadata": {},
   "source": [
    "### Run Operator with `Pipeline`\n",
    "\n",
    "When an operator is uploaded to the Towhee hub or registered with @register, we can call the operato directly on a Pipeline:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "268722a5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table style=\"border-collapse: collapse;\"><tr><th style=\"text-align: center; font-size: 130%; border: none;\">url</th> <th style=\"text-align: center; font-size: 130%; border: none;\">image</th></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">1.jpg</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \"><img src=\"\" 128 = \"128px\" 128 = \"128px\" style = \"float:left; padding:2px\"></td></tr></table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<table style=\"border-collapse: collapse;\"><tr><th style=\"text-align: center; font-size: 130%; border: none;\">num</th> <th style=\"text-align: center; font-size: 130%; border: none;\">processed_num</th></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">1</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">3</td></tr></table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "p = (\n",
    "\tpipe.input('url')\n",
    "\t\t.map('url', 'image', ops.image_decode.cv2('rgb'))\n",
    "\t\t.output('url', 'image')\n",
    ")\n",
    "DataCollection(p('1.jpg')).show()\n",
    "\n",
    "p = (\n",
    "\tpipe.input('num')\n",
    "\t\t.map('num', 'processed_num', ops.add_x(x=2))\n",
    "\t\t.output('num', 'processed_num')\n",
    ")\n",
    "DataCollection(p(1)).show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1c86d9ef",
   "metadata": {},
   "source": [
    "## Complete Example"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c36f9e06",
   "metadata": {},
   "source": [
    "Here we show a complete example of towhee `Pipeline` that implements a image embedding task"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "2b707c3c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table style=\"border-collapse: collapse;\"><tr><th style=\"text-align: center; font-size: 130%; border: none;\">urls</th> <th style=\"text-align: center; font-size: 130%; border: none;\">images</th> <th style=\"text-align: center; font-size: 130%; border: none;\">embeddings</th></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">1.jpg</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \"><img src=\"\" 128 = \"128px\" 128 = \"128px\" style = \"float:left; padding:2px\"></td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[0.010864214, 0.0, 0.2484118, ...] shape=(2048,)</td></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">2.jpg</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \"><img src=\"\" 128 = \"128px\" 128 = \"128px\" style = \"float:left; padding:2px\"></td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[0.0, 0.0, 0.0, ...] shape=(2048,)</td></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">3.jpg</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \"><img src=\"\" 128 = \"128px\" 128 = \"128px\" style = \"float:left; padding:2px\"></td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[0.0002684289, 0.0, 0.17076984, ...] shape=(2048,)</td></tr></table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "emb_pipe = (\n",
    "\tpipe.input('urls')\n",
    "\t\t.flat_map('urls', 'urls', lambda x: x)\n",
    "\t\t.map('urls', 'images', ops.image_decode.cv2('rgb'))\n",
    "\t\t.map('images', 'embeddings', ops.image_embedding.timm(model_name='resnet50'))\n",
    "\t\t.output('urls', 'images', 'embeddings')\n",
    ")\n",
    "\n",
    "res = emb_pipe(['1.jpg', '2.jpg', '3.jpg'])\n",
    "DataCollection(res).show()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "base",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  },
  "vscode": {
   "interpreter": {
    "hash": "f7dd10cdbe9a9c71f7e71741efd428241b5f9fa0fecdd29ae07a5706cd5ff8a2"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
